The ISLE Corpus of Non-Native Spoken English

For the purpose of developing pronunciation training tools for second language learning a corpus of non-native speech data has been collected, which consists of almost 18 hours of annotated speech signals spoken by Italian and German learners of English. The corpus is based on 250 utterances selected from typical second language learning exercises. It has been annotated at the word and the phone level, to highlight pronunciation errors such as phone realisation problems and misplaced word stress assignments. The data has been used to develop and evaluate several diagnostic components, which can be used to produce corrective feedback of unprecedented detail to a language learner. 1 This research has been supported by the European Commission under the 4 framework of the Telematics Application Programme (Language Engineering Project LE4-8353). Introduction Project ISLE (Interactive Spoken Language Education) has the goal of integrating state-of-the-art Hidden Markov Model [HMM] speech recognition technologies into a computer-based package for intermediate level learners of English. The use of speech recognition [SR] will allow students to use spoken language as the most natural form of communication. More importantly it allows for on-line diagnosis and correction of both the communicative and grammatical adequacy of the spoken utterance and possible pronunciation errors made when speaking. The first goal is achieved in a customary way by prompting students with a small set of options they can select from. A low perplexity speech recognizer checks whether the student’s selection was an appropriate one. Speaker adaptation techniques are used to compensate for accented, non-native speech. Afterwards, custom designed components of the ISLE system are invoked to locate and describe phoneand stress-level pronunciation errors in the utterance (Herron et al. 1999). Thus, the system is in a position to produce detailed feedback to the student and to offer tailored practice for the errors encountered. The range of oral activities the student is engaged in includes reading exercises, producing minimal pairs, selecting a item from a list of options, and combining items from different selections. Although the technologies used and developed are theoretically valid for any language pairs, ISLE is focusing on Italian and German learners of English. Purpose/Goals To support the development of pronunciation training tools a corpus of non-native speech was required for three reasons: 1. to train the parameters and rules used in the recognition and diagnosis systems; 2. to test the performance of the system on a known data set; and 3. to evaluate the contribution of speaker adaptation for improving the reliability of the native British English recognizer. While the last function requires only a word-level transcription the first two also demand that the corpus be annotated at the phoneand stress-level for pronunciation errors. Additionally, it was desirable to test the system on a variety of exercises of various complexities (or perplexities, more specifically), since the actual system was planned to have both simple and complex exercises. The language material for testing the speaker adaptation was chosen from a non-fictional, autobiographical text describing the ascent of Mount Everest (Hunt, 1996). It was selected so that speakers/readers would not have to deal with reported speech or foreign words, which may cause them to alter their pronunciation. Approximately 1300 words of the text (82 sentences) were chosen, to be read by each speaker. To test the recognition and error diagnosis capabilities a different kind of data was collected with the intention of capturing typical pronunciation errors made by non-native speakers of English in controlled language learning situations. Therefore, the constraints on this kind of data come firstly from the exercise types which have been identified as being important by an initial user survey, and secondly from the tasks complexity for which a sufficiently high recognition accuracy can be expected. The linguistic complexity for the speech recognizer is restricted by assuming that a mini-grammar can be written by the courseware author for the intended domain. Initially a perplexity of 6-10 was considered acceptable for exercises where alternative words and expressions were to be chosen from a given list. This part of the material consists of approximately 1100 words contained in 164 phrases.